Issue link(s): Resolves #5601

Summary - What I changed

This PR fixes several bugs in how related_integrations.version is computed at export time. All of them affected the value written into prebuilt rules; none change ES|QL validation (that still uses find_latest_compatible_version / patch inference from #6251). Addressing the initial bug is a smaller PR, but through testing, we identified several other issues predominantly with ESQL (given we do not have an ESQL parser), that expands the scope of the changes.

Bugs fixed

1. Stack-dependent export ([Bug] Improper prebuilt rule version usage in security_detection_engine package versions #5601). find_least_compatible_version picked a single ^X.Y.Z from the build-time stack version. The same rule id could ship ^8.2.0 on an 8.19 backport and ^9.0.0 on 9.x while the prebuilt rule version integer stayed the same, which triggered false integration version-mismatch warnings after upgrade.

2. Schema-blind OR ranges. Rebased onto [Bug] ESQL Remote Validation Data Stream and Patch Version Validation #6251 so export respects integration-schemas.json.gz: data streams introduced mid-package (for example aadgraphactivitylogs in azure 1.37.0) get a schema floor in the OR range, not just Kibana manifest compatibility.

3. Phantom anchors from unshipped stack lines. Walking every Kibana major in the manifest could emit legacy anchors such as ^0.0.2 from azure packages that still declare ^7.9.0. Export now intersects with stack majors from the stack-schema-map backport lines we actually ship rules to.

4. Incomplete stack-major coverage. Bounded ranges like >=8.12.0 <9.1.0 omitted major 9; unbounded >= ranges stopped at the lower bound only. Fixed with per-clause major overlap and a cap for open-ended ranges.

5. Duplicate ES|QL related_integrations rows. ES|QL rules tagged [metadata] integration = ["azure"] with a query filter on azure.signinlogs (or similar) got both a spurious metadata-only package row and the data-stream row. Kuery/EQL were unaffected. Fixed with a prefix check so the metadata row is skipped when the query already names a dataset for that package.

Implementation

• Replaces find_least_compatible_version with find_compatible_version_range, which returns a stack-invariant OR'd caret range (for example ^8.2.0 || ^9.0.0 || ^10.0.0 for endpoint) plus a forward sentinel for the next integration major.
• policy_templates are unioned across manifest-backed anchors, not parsed from the OR string.
• UNKNOWN_PACKAGE_INTEGRATION constant; tuple anchors on CompatibleVersionRange.

Performance: the new path resolves more stack lines per package. Informal local timing was roughly 2 to 9x median vs the old function on endpoint / aws / windows manifests, still sub-10ms per call and once per rule at package build. Not expected to move build-release wall time in a meaningful way.

Follow-up: version.lock bumps for affected rules are planned separately before the next package release (per review).

How To Test

make deps
make pytest
python -m detection_rules view-rule rules_building_block/defense_evasion_download_susp_extension.toml

Spot check export: rule 8d366588-cbd6-43ba-95b4-0971c3f906e5 should show ^8.2.0 || ^9.0.0 || ^10.0.0 for endpoint on every release line.

  "related_integrations": [
    {
      "package": "endpoint",
      "version": "^8.2.0 || ^9.0.0 || ^10.0.0"
    }
  ],

Export toml and import into kibana to ensure it applies successfully.

uv run python -m detection_rules export-rules-from-repo -f rules_building_block/defense_evasion_download_susp_extension.toml

20260528T154018L.ndjson.txt

Backport validation: cherry-pick this PR onto each release branch (or apply the changed files from the branch), then run the same pytest commands. No version.lock changes required for that pass.

$ pytest -q
FAILED tests/test_all_rules.py::TestRuleMetadata::test_rule_change_has_updated_date
1 failed, 221 passed, 19 skipped, 2 subtests passed in 69.53s (0:01:09)

# same failure on vanilla origin/8.19 (diffs rules/ vs origin/main, not introduced here)

$ pytest -q -k "not test_rule_change_has_updated_date"
221 passed, 19 skipped, 1 deselected, 2 subtests passed in 65.20s (0:01:05)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.51s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

$ pytest -q
222 passed, 19 skipped, 2 subtests passed in 82.76s (0:01:22)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.54s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

$ pytest -q
222 passed, 19 skipped, 2 subtests passed in 96.19s (0:01:36)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.52s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

$ pytest -q
222 passed, 19 skipped, 2 subtests passed in 106.83s (0:01:46)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.50s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

Checklist

• Added a label for the type of pr: bug
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation